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DETAILED ACTION 
Response to Arguments 

Applicant's arguments, see Remarks pages 1-3, filed 25 May 2005, with respect 
to the rejection(s) of claim(s) 1. 3, 5, 7-12, 17, 19, 20. and 24 under 35 U.S.C. 102(b) 
and 35 U.S.C. 103(a) have been fully considered and are persuasive in view of 
applicant's amendments, since applicant has canceled several claims and incorporated 
their limitations into the independent claim, and otherwise changed the claims 
suffidently to require new rejections (specifically moving from 35 U.S.C. 102(b) on the 
independent claims to 35 U.S.C. 103(a). Therefore, the rejection has been withdrawn. 

The rejections of claims 2, 4, 6. 13-16, 18. and 21-23 under 35 U.S.C. 102(b) and 
35 U.S.C. 103(a) are withdrawn since the claims have been canceled. 

However, upon further consideration, a new ground(s) of rejection is made in 
view of various references as below. 

Examiner does not regard the changing of the word 'orientation' to 'orientation' in 
the claims as changing claim scope. Firstly, applicant does not specify that such a 
change in wording was made in order to differentiate and/or alter claim scope. 
Secondly, the Chen reference as below specifically teaches an orientation sensor on 
the video camera anyway (3:65-4:1 0). Lastly, two definitions of orientation from 
dictionary.com are: 

1 . The orientation of an aircraft's axes relative to a reference line or plane, such as 
the horizon. 

2. The orientation of a spacecraft relative to its direction of motion. 
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The definitions of the term therefore Indicate that the word Is synonymous with and 
equivalent to the tern "orientation" In any case. 

The other changes, such as the word 'creating' to 'generating' require further 
search and consideration, and as such justify the shifting of the rejections from 35 
U.S.C. 102(b) to 36 U.S.C. 103(a). 

Applicant's arguments fail to comply with 37 CFR 1 .1 1 1(b) because they amount 
to a general allegation that the claims define a patentable invention without specifically 
pointing out how the language of the claims patentably distinguishes them from the 
references. Specifically, applicant incorporated the limitations of claim 2 into claim 1 . 
While examiner concedes that Chen does not in fact teach the added limitations, this 
point was never questioned, as examiner utilized a rejection under 35 U.S.C. 1 03(a) 
against daim 2 with two references. Applicant showed why Chen does not teach the 
added limitations per se, but failed to in any way address the combination with the 
Schneider reference, as applied to the rejection of claim 2 under 35 U.S.C. 103(a). 
Therefore, a final rejection is proper and the original grounds of rejection (as modified to 
fit the new claim structure) are applied. 

In response to applicant's arguments against the references individually, one 
cannot show nonobviousness by attacking references individually where the rejections 
are based on combinations of references. See In re Keller, 642 F.2d 413, 208 
USPQ 871 (CCPA 1 981 ); In re Merck & Co., 800 F.2d 1 091 , 231 USPQ 375 (Fed. CIr. 
1986). 
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Applicant's assertion that Schneider does not teach template matching is 
incon-ect. Schneider as stated below clearly teaches template matching (5:50-67, 
particularly 5:64-66). Since the segmentation could be done using template matching, 
obviously that constitutes a detecting unit that detects the position of said index (e.g. the 
fiducial marks) by segmentation via template matching, which Schneider teaches is 
well known in the art. The system of Schneider must inherently function in the manner 
of the system of applicant (see the example with Okumura below). A template-matching 
system always takes the image of a template (what applicant refers to as the template 
image of said index)(Figure 4 of Okumura) and compares it to the overall acquired 
linage (e.g. the target image) to localize and obtain only the image of the desired region 
(the template area). 

As an example, Okumura (US 5,878,156) teaches the use of template-matching 
techniques in 6:30-55, where in Figure 4 the use of template matching is shown, where 
a rectangular face-template 10 is selected from a target image as in Figure 5, and then 
the eye region template is used to extract the eyes from that image. Okumura is a good 
example of how template-matching techniques inherently work. 

Also, it is noted for purposes of appeal that claim 26 is in all probability non- 
statutory. It is further noted that such a new grounds of rejection will be added to the 
Examiner's Answer, since the Board would otherwise remand such a case for the 
addition of such grounds. Applicant is advised that such a rejection may be avoided by 
filing an amendment under Rule 312 combining claims 26 and 27. If any other matter is 
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submitted with such an amendment prior to the filing of the appeal brief, such an 

amendment will not be entered. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

Claims 1 , 9, 17. 24, 26-27. and 28 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Chen in view of Schneider. 
As to claim 1, 

Chen et al. discloses the system and method as claimed in claims 1 and 24. 
Chen et al. describes a surgical targeting system that involves displaying an image 
created by intemiixing a virtual image with a real image. Column 9, lines 38-43. 
describes aligning a real image taken from a video camera with a virtual image by 
holding the camera in a fixed position. "A first, and generally more preferable, 
technique involves holding the position of video camera 45 (and hence real image 55) 
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constant and moving the position of the "virtual object" or the "virtual camera" by means 
of apparatus 57 until the virtual image 60 is brought into registration with real image 55." 
Column 1 1 , lines 7-11, states, "More particularly, and looking now at FIG. 8, there is 
shown a video camera 45, an anatomical structure 60, and a tracker system 65. 
Tracker system 65 comprises a tracker 70 which is attached to video camera 45, and a 
tracker base 75 which defines the coordinate system of the tracker system." Thus, 
Chen et al. teaches of an orientation sensor for measuring the orientation of the video 
camera (see 3:65-4:10). Lines 12-17 disclose matrix transformations from the patient 
to the camera, the camera to the tracker base, and the patient to the tracker base. 
Thus, calculation information is stored in the matrix transfonnations to calculate the 
orientation of the patient on the basis of an output from the tracker system. Column 6, 
lines 61 - 67, and column 7, lines 1 - 5, describe the use of an imaging machine to 
provide two-dimensional images of a patient's anatomical structure and storing the 
images in a patient-specific database. Column 8, lines 10-46, discloses the 
placement of virtual planning markers in the patient-specific images by a physician. 
Column 9, lines 43 - 47, describes a detecting algorithm to perform a template 
matching process between planning markers of a virtual image and a target image. 
"This can be done automatically by having image generator 30 use a search algorithm 
to match the virtual image to the real image, in which case apparatus 57 includes 
computer hardware and software of the sort well known in the art to cause image 
generator 30 to work through a search algorithm to match the virtual image to the real 
image." Column 12, lines 26 - 37, discloses detecting the position of an anatomical 
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structure in a real image by perfomning a data matching procedure to correlate sampled 
points with patient-specific images. "It is also anticipated that one could use a tracked 
surgical instrument 85 to determine the location of anatomical structure 60. This can be 
accomplished by using the tracked surgical instrument 85 to engage known fiducial 
points on the anatomical structure. Alternatively, a tracked surgical instrument 85 can 
be used to sample multiple surface points located on the anatomical structure and then 
use a data matching procedure to correlate the sampled points wjth either patient- 
specific database 10 or patient-specific 3-D computer model 15." Column 1 1 , lines 18 - . 
29, describe an updating of the calculation information on the basis of a detected 
position of a marker in the patient image then calculating the orientation of the patient in 
relation to the tracker system on the basis of known values and the updated calculation 
infonnation from the matrix transformations. "M.sub.CT is known from the tracker 
system. Furthemnore, once the virtual image generated by image generator 30 has 
been placed in registration with the real image generated by camera 45, the virtual 
camera position will be known relative to the virtual anatomical structure, and hence the 
real camera position will be known relative to the real anatomical structure. Thus, real 
matrix M.sub.PC will also be known. In addition, since M.sub.CT and M.sub.PC are 
then both known, it is possible to solve for M.sub.PT. Accordingly, the position of 
anatomical structure 60 will then also be known within the relative coordinate system 
defined by the tracker system." 
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The template matching operation is inherent to Schneider, as discussed above in 
the Response to Arguments section, the relevant portion of \A^ich is incorporated by 
reference. 

Chen et al. discloses the device of claim 1 except wherein said target image 
setting unit obtains a predicted position of the index in the picked-up image employing 
said measured value and said calculation information stored in said storage unit and 
creates an image with a peripheral area around said prediction position in said picked- 
up image subjected to a rotational process on the basis of a rotational angle in a roll 
direction of said image pick-up device derived from said measured value. The invention 
of Schneider teaches of obtaining images in a pre-procedural time and segmenting 
them into sub-objects that have defined boundaries, shapes, and positions within the 
overall image. Column 8, lines 36 - 52, describes segmenting the images while they 
are digitally analyzed to identify predefined fiducial markers by means of feature 
extraction, edge detection, region growing, boundary analysis, template matching, etc. 
Column 9, lines 4-10, states, "Particularly likely to be useful in the future are those 
statistical segmentation techniques that assign to each point a certain degree of 
probability as to whether or not it is a part of a given segmented object. That probability 
is based upon a variety of factors including pixel intensity and location with respect to 
other pixels of given qualities. Once probabilities of each pixel have been determined, 
assessments can be made of the pixels as a group, and segmentation can be achieved 
with improved accuracy." Additionally, column 10, lines 4-15, teaches of rotating 
captured images so as to align the images, according to the fiducial markers, to a 
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corresponding position orientation. It would have been obvious to one having ordinary 
skill in the art at the time the invention was made to modify the invention of Chen et al. 
to include segmenting the captured virtual images so as to include a peripheral area 
around a predicted position of the index. One would have been motivated to make such 
a modification to the invention of Chen so that upon performing template matching 
between virtual and real images, the speed of identifying marker matches is increased 
by bounding a search area within a smaller particular region of the overall image. It 
would have been further obvious to one having ordinary skill in the art at the time the 
invention was made to modify the invention of Chen to include subjecting the captured 
images to rotational process. One would have been motivated to make such a 
modification to the invention of Chen so that while perfonning template matching 
between a virtual and a real image, the orientations of the images may be aligned so as 
to properly line up their fiducial markers. 
As to claim 9, 

Chen et al. discloses the device of daim 9. Column 1 1 , lines 12-29. discloses 
calculation information in the form of matrix transformations for calculating the position 
and attitude of the patient on the basis of the measured value and position information 
of the pick-up visual point of the camera in regard to the tracker system. "In this setting. 
M.sub.PC can be considered to represent the matrix transfonnation from the patient's 
anatomical structure 60 to camera 45; M.sub.CT can be considered to represent the 
matrix transformation from camera 45 to tracker base 75; and M.sub.PT can be 
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considered to represent the matrix transfonnation from anatomical structure 60 to 
tracker base 75. 

M.sub.CT is known from the tracker system. Furthermore, once the virtual image 
generated by image generator 30 has been placed in registration with the real image 
generated by camera 45, the virtual camera position will be known relative to the virtual 
anatomical structure, and hence the real camera position will be known relative to the 
real anatomical structure. Thus, real matrix M.sub.PC will also be known. In addition, 
since M.sub.CT and M.sub.PC are then both known, it is possible to solve for M.sub.PT. 
Accordingly, the position of anatomical structure 60 will then also be known within the 
relative coordinate system defined by the tracker system." Therefore, the calculation 
information is updated once the virtual image is placed in registration with the real 
image generated by the camera and the matrix transformations are fomned. 

As to claim 17, 

Chen et al. discloses the device of claim 17. Column 10, lines 1-15, states, 
"Once virtual image 50 has been placed into proper registration with real image 55, 
image generator 30, video camera 45 and video mixing device 35 can be used to 
present the virtual and real images on display 40 in various presentation formats so as 
to facilitate a particular medical procedure. In particular, one can use Image generator 
30, video camera 45 and video mixing device 35 to superimpose a virtual image 
(generated from patient-specific 3-D computer model 1 5) against a real image 
(generated by video camera 45), with image generator 30 being directed to modify the 
virtual image so as to expose one or more of the virtual planning mariners 25 present in 
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patient-specific 3-D computer model 15, whereby the anatomy highlighted by virtual 
planning markers 25 will be brought quickly to the attention of the physician." Thus, real 
image is a visual point of the visual camera. Additionally, column 9, lines 43 - 49, notes 
a search algorithm to match the virtual and real images. "This can be.done 
automatically by having image generator 30 use a search algorithm to match the virtual 
image to the real image, in which case apparatus 57 includes computer hardware and 
software of the sort well known in the art to cause image generator 30 to work through a 
search algorithm to match the virtual image to the real image." The change in wording 
from 'display means' to 'display device' from the incorporated limitations from the former 
claim 18 as well as other cosmetic changes does not change claim scope. 
As to claim 24. 

Chen et al. discloses the method as claimed in claim 24. Chen et al. describes a 
surgical targeting system that involves displaying an image created by intermixing a 
virtual image with a real image. Column 9. lines 38 - 43, describes aligning a real 
image taken from a video camera with a virtual image by holding the camera in a fixed 
position. "A first, and generally more preferable, technique involves holding the position 
of video camera 45 (and hence real image 55) constant and moving the position of the 
"virtual object" or the "virtual camera" by means of apparatus 57 until the virtual image 
50 is brought into registration with real image 55." Column 1 1 . lines 7-11. states, 
"More particularly, and looking now at FIG. 8, there is shown a video camera 45,. an 
anatomical stmcture 60. and a tracker system 65. Tracker system 65 comprises a 
tracker 70 which is attached to video camera 45, and a tracker base 75 which defines 
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the coordinate system of the tracker system." Thus, Chen et al. teaches of an attitude 
sensor for measuring the attitude of the video camera. Lines 12-17 disclose matrix 
transformations from the patient to the camera, the camera to the tracker base, and the 
patient to the tracker base. Thus, calculation information is stored in the matrix 
transformations to calculate the attitude of the patient on the basis of an output from the 
tracker system. Column 6, lines 61 - 67, and column 7, lines 1 - 5, describe the use of 
an imaging machine to provide two-dimensional images of a patient's anatomical 
structure and storing the images in a patient-specific database. Column 8, lines 10 - 
46, discloses the placement of virtual planning markers in the patient-specific images by 
a physician. Column 9, lines 43-47, describes a detecting algorithm to perfonn a 
template matching process between planning markers of a virtual image and a target 
image. "This can be done automatically by having image generator 30 use a search 
algorithm to match the virtual image to the real image, in which case apparatus 57 
includes computer hardware and software of the sort well known in the art to cause 
image generator 30 to work through a search algorithm to match the virtual image to the 
real image." Column 12, lines 26 - 37, discloses detecting the position of an anatomical 
structure in a real image by performing a data matching procedure to correlate sampled 
points with patient-specific images. "It is also anticipated that one could use a tracked 
surgical instrument 85 to determine the location of anatomical structure 60. This can be 
accomplished by using the tracked surgical instrument 85 to engage known fiducial 
points on the anatomical structure. Alternatively, a tracked surgical instrument 85 can 
be used to sample multiple surface points located on the anatomical structure and then 
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use a data matching procedure to correlate the sampled points with either patient- 
specific database 10 or patient-specific 3-D computer model 15." Column 1 1 , lines 18 - 
29, describe an updating of the calculation infonmation on the basis of a detected 
position of a marker in the patient image then calculating the attitude of the patient in 
relation to the tracker system on the basis of known values and the updated calculation 
infomiation from the matrix transformations. "M.sub.CT is known from the tracker 
system. Furthermore, once the virtual image generated by image generator 30 has 
been placed in registration with the real image generated by camera 45, the virtual 
camera position will be known relative to the virtual anatomical structure, and hence the 
real camera position will be known relative to the real anatomical structure. Thus, real 
matrix M.sub.PC will also be known. In addition, since M.sub.CT and M.sub.PC are 
then both known, it is possible to solve for M.sub.PT. Accordingly, the position of 
anatomical structure 60 will then also be known within the relative coordinate system 
defined by the tracker system." 

As discussed above, the template matching is inherent to Schneider. 

The template matching operation is inherent to Schneider, as discussed above in 
the Response to Arguments section, the relevant portion of which is incorporated by 
reference. 

Chen et al. discloses the device of claim 24 except template matching and 
certain other minor details. The invention of Schneider teaches of obtaining images in a 
pre-procedural time and segmenting them into sub-objects that have defined 
boundaries, shapes, and positions within the overall image. Column 8, lines 36 - 52, 



Application/Control Number: 10/644,829 Page 14 

Art Unit: 2672 

describes segmenting the images while they are digitally analyzed to identify predefined 
fiducial markers by means of feature extraction, edge detection, region growing, 
boundary analysis, template matching, etc. Column 9, lines 4-10, states, "Particularly 
likely to be useful in the future are those statistical segmentation techniques that assign 
to each point a certain degree of probability as to whether or not it is a part of a given 
segmented object. That probability is based upon a variety of Actors including pixel 
intensity and location with respect to other pixels of given qualities. Once probabilities 

/ 

of each pixel have been determined, assessments can be made of the pixels as a 
group, and segmentation can be achieved with improved accuracy." Additionally, 
column 10, lines 4-15, teaches of rotating captured images so as to align the images, 
according to the fiducial markers, to a corresponding position orientation. It would have 
been obvious to one having ordinary skill in the art at the time the Invention was made 
to modify the invention of Chen et al. to include segmenting the captured virtual images 
so as to include a peripheral area around a predicted position of the index. One would 
have been motivated to make such a modification to the invention of Chen so that upon 
performing template matching between virtual and real images, the speed of identifying 
marker matches is Increased by bounding a search area within a smaller particular 
region of the overall image. It would have been further obvious to one having ordinary 
skill in the art at the time the invention was made to modify the invention of Chen to 
include subjecting the captured images to rotational process. One would have been 
motivated to make such a modification to the invention of Chen so that while perfomiing 
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template matching between a virtual and a real Image, the orientations of the images 
may be aligned so as to property line up their fiducial markers. 

As to claims 26 and 27, 

Chen et al. discloses the program code of claim 26 and the storage medium of 
claim 27. Figure 10 shows an overall flow diagram of the invention of Chen. A program 
module is shown that runs an algorithm to align the real and virtual images. Thus, the 
algorithm to be mn is interpreted as program code for executing the image processing 
method. Column 9, lines 38 - 49, describes the use of the algorithm including computer 
hardware and software such that the program code is stored on the hardware medium. 

As to claim 28, this is merely a broadened version of claims 1 and 24, with the 
preamble encompassing some of the limitations (notably the first clause and the 
template matching portion of the detecting step), the orientation step, the target image- 
creating step of daim 1 , and the detecting step. 

Chen et al. discloses the device of claim 28, except wherein said target image 
setting unit obtains a predicted position of the index in the picked-up image employing 
said measured value and said calculation information stored in said storage unit and 
creates an image with a peripheral area around said prediction position in said picked- 
up image subjected to a rotational process on the basis of a rotational angle in a roll 
direction of said image pick-up device derived from said measured value. The invention 
of Schneider teaches of obtaining images in a pre-procedural time and segmenting 
them into sub-objects that have defined boundaries, shapes, and positions within the 
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overall Image. Column 8. lines 36 - 52, describes segmenting the images while they 
are digitally analyzed to identify predefined fiducial markers by means of feature 
extraction, edge detection, region growing, boundary analysis, template matching, etc. 
Column 9, lines 4-10, states, "Particularly likely to be useful in the future are those 
statistical segmentation techniques that assign to each point a certain degree of 
probability as to whether or not it is a part of a given segmented object. That probability 
is based upon a variety of factors including pixel intensity and location with respect to 
other pixels of given qualities. Once probabilities of each pixel have been determined, 
assessments can be made of the pixels as a group, and segmentation can be achieved 
with improved accuracy." Additionally, column 10, lines 4-15, teaches of rotating 
captured images so as to align the images, according to the fiducial markers, to a 
conresponding position orientation. It would have been obvious to one having ordinary 
skill in the art at the time the invention was made to modify the invention of Chen et al. 
to include segmenting the captured virtual images so as to include a peripheral area 
around a predicted position of the index. One would have been motivated to make such 
a modification to the invention of Chen so that upon perfonning template matching 
between virtual and real images, the speed of identifying mariner matches is increased 
by bounding a search area within a smaller particular region of the overall image. It 
would have been further obvious to one having ordinary skill in the art at the time the 
invention was made to modify the invention of Chen to include subjecting the captured 
images to rotational process. One would have been motivated to make such a 
modification to the invention of Chen so that while performing template matching 
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between a virtual and a real image, the orientations of the images may be aligned so as 
to properly line up their fiducial markers. 

Claims 3, 10-12. and 20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Chen in view of Schneider as applied to claim 1 and further in view of 
Rallison et al. 

As to claim 3, 

Chen and Schneider disclose the device of claim 3 except wherein said 
calculation information is the correction infonmation to con-ect for an error In the 
measured value of attitude measured by said attitude sensor of the image pick-up visual 
point of said image pick-up device, and said calculation unit calculates attitude of said 
measurement object on the basis of the measured value and correction information. 
The invention of Rallison et al. teaches of tracking the position and attitude of an image 
generator mounted on the head of a person. Column 19. lines 11-26. teaches of the 
various types of tracker systems including magnetic and inertial sensor systems. 
Columns 21 , 22. and 23 explain obtaining position and attitude orientation with the two 
systems. Column 24. lines 37 - 67, and column 25. lines 23 - 62, describe con-ecting 
for an en-or in the measured values of attitude and position taken from the sensor 
systems. It would have been obvious to one having ordinary skill in the art at the time 
the invention was made to modify the invention of Chen et al. to include using an inertial 
sensor system as the tracker system for the video camera. One would have been 
motivated to make such a modification to Chen since inertial sensors such as rate gyros 
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will have the advantage of being substantially immune to magnetic perturbations that 
may be present in a medical operating atmosphere. It would have been further obvious 
to one having ordinary skill in the art at the time the invention was made to correct for 
an en-or in the measured value of attitude information of the inertial sensors for use in 
the matrix transformation calculations of Chen. One would have been motivated to 
make such a modification to the invention of Chen so that the calculated position and 
attitude information of the camera and patient remain accurate while performing a 
medical procedure. 

As to claims 10, 1 1, and 12, 

Chen et al. discloses the device of claims 1 0, 1 1 , and 12. Column 1 1 , lines 1 2 - 
29, discloses calculation information in the form of matrix transformations for calculating 
the position and attitude of the patient on the basis of the measured value and position 
information of the pick-up visual point of the camera in regard to the tracker system. "In 
this setting, M.sub.PC can be considered to represent the matrix transformation from 
the patient's anatomical structure 60 to camera 45; M.sub.CT can be considered to 
represent the matrix transformation from camera 45 to tracker base 75; and M.sub.PT 
can be considered to represent the matrix transfomiation from anatomical structure 60 
to tracker base 75. 

M.sub.CT is known from the tracker system. Furthermore, once the virtual image 
generated by image generator 30 has been placed in registration with the real image 
generated by camera 45, the virtual camera position will be known relative to the virtual 
anatomical structure, and hence the real camera position will be known relative to the 
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real anatomical structure. Thus, real matrix M.sub.PC will also be known. In addition, 
since M.sub.CT and M.sub.PC are then both known, it is possible to solve for M. sub. FT. 
Accordingly, the position of anatomical stmcture 60 will then also be known within the 
relative coordinate system defined by the tracker system." Therefore, the calculation 
information is updated once the virtual image is placed in registration with the real 
image generated by the camera and the matrix transformations are fomried. 
Additionally, column 12, lines 35 - 37, describes using a least squares fit to correlate 
the sampled points with the patient-specific images. Thus, a typical value is used that 
incorporates a dislocation value between the predicted position and the detected 
position of a plurality of markers from a real and virtual image. 
As to claim 20, 

Chen et al. discloses the device of claim 20, see description according to claim 1 , 
the rejection to which is incorporated by reference, except wherein a target image is 
created having a peripheral area around a predicted position in a picked-up image 
subjected to a rotational process on the basis of a rotational angle in a roll direction of 
said image pick-up device derived firom said measured value. Chen et al. teaches in 
column 1 1 , lines 3 - 1 1 , of a tracker system attacked to a video camera such that the 
position of the camera will be predetermined by the tracker system. Schneider teaches 
of obtaining images in a pre-procedural time and segmenting them into sub-objects that 
have defined boundaries, shapes, and positions within the overall image. Column 8, 
lines 36 - 52, describes segmenting the images while they are digitally analyzed to 
identify predefined fiducial markers by means of feature extraction, edge detection. 
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region growing, boundary analysis, template matching, etc. Column 9, lines 4-10, 
states, "Particularly likely to be useful in the future are those statistical segmentation 
techniques that assign to each point a certain degree of probability as to whether or not 
it is a part of a given segmented object. That probability is based upon a variety of 
factors including pixel intensity and location with respect to other pixels of given 
qualities. Once probabilities of each pixel have been determined, assessments can be 
made of the pixels as a group, and segmentation can be achieved with improved 
accuracy." Additionally, column 10, lines 4-15, teaches of rotating captured images so 
as to align the images, according to the fiducial markers, to a con-esponding position 
orientation. Thus, the position of the area to be segmented is determined by a 
statistical segmentation technique while the video camera is at a predetermined 
position. It would have been obvious to one having ordinary skill in the art at the time 
the invention was made to modify the invention of Chen et al. to include segmenting the 
captured virtual images so as to include a peripheral area around a predicted position of 
the index. One would have been motivated to make such a modification to the invention 
of Chen so that upon performing template matching between virtual and real images, 
the speed of identifying marker matches is increased by bounding a search area within 
a smaller particular region of the overall image. It would have been further obvious to 
one having ordinary skill in the art at the time the invention was made to modify the 
invention of Chen to include subjecting the captured images to rotational process. One 
would have been motivated to make such a modification to the invention of Chen so that 
while performing template matching between a virtual and a real image, the orientations 



Application/Control Number: 1 0/644,829 Pag© 21 

Art Unit: 2672 

of the images may be aligned so as to properly line up their fiducial markers. Motivation 
for combination with Schneider is taken from the rejection to claim 1 . which has been 
incorporated by reference. 

Claims 5. 7-8, and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Chen in view of Schneider, and further in view of Rallison. 
As to claim 5, 

Chen et al. and Schneider discloses the Invention of claim 5 except wherein said 
calculation information is the conrection information to correct for an error in the 
measured value of attitude measured by said attitude sensor and the position 
infomiation of the image pick-up visual point of said image pick-up device, and said 
calculation unit calculates the position and attitude of said measurement object on the 
basis of the measured value, correction information, and the position information. The 
invention of Rallison et al. teaches of tracking the position and attitude of an image 
generator mounted on the head of a person. Column 19, lines 1 1 - 26, teaches of the 
various types of tracker systems including magnetic and inertial sensor systems. 
Columns 21, 22, and 23 explain obtaining position orientation with the two systems. 
Column 24, lines 37 - 67, and column 25, lines 23 - 62, describe correcting for an error 
in the measured values of attitude and position taken from the sensor systems. It would 
have been obvious to one having ordinary skill in the art at the time the invention was 
made to modify the invention of Chen et al. to include using an inertial sensor system as 
the tracker system for the video camera. One would have been motivated to make such 
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a modification to Chen since inertial sensors sucli as rate gyros will have the advantage 
of being substantially immune to magnetic perturbations that may be present in a 
medical operating atmosphere. It would have been further obvious to one having 
ordinary skill in the art at the time the invention was made to correct for an error in the 
measured value of attitude and position information of the inertial sensors for use in the 
matrix transformation calculations of Chen. One would have been motivated to make 
such a modification to the invention of Chen so that the calculated position and attitude 
information of the camera and patient remain accurate while performing a medical 
procedure. 

As to claim 7, 

Chen, Schneider, and Rallison teach of the device of claim 7. Chen and 
Schneider in view of Rallison teach of updating the position information with regard to a 
series of matrix transformations in column 1 1 , lines 7 - 30. Additionally, column 9, lines 
38 - 49, describe a search algorithm for automatically bringing a virtual image into 
registration with a real image. Column 10, lines 38 - 52, describes reestablishing the 
registration between a real image and a virtual image if the video camera position is 
moved. Thus, Chen, Schneider, and Rallison teach of updating position information. It 
would have been obvious to one having ordinary skill in the art at the time the invention 
was made to modify the invention of Chen, Schneider, and Rallison so that the position 
information is only updated in two directions and not in a depth direction if the action is 
so desired by a user while reestablishing a registration between a real image and a 
virtual image. One would have been motivated to make such a modification to the 
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invention of Chen. Schneider, and Rallison so that if a user does not require an updated 
value in a depth direction, the computing system does not waste utilization time and 
resources calculating an unnecessary value. 
As to claim 8, 

Chen. Schneider, and Rallison explicitly disclose the device of claim 8 except 
wherein the correction infonnation is the infomnation to correct for an error in the 
azimuth direction among the measured values of the attitude measured by the attitude 
sensor. In column 23. lines 60 - 67. and column 24. lines 1 - 23. Rallison additionally 
teaches of sensor drift, thus causing an error in the measured values of roll, pitch, and 
yaw. 

The definition of the term 'yaw' is "to turn about the vertical axis. Used of an 
aircraft, spacecraft, projectile, etc." The definition of the term "azimuth" is "the horizontal 
angular distance from a reference direction, to the point where a vertical circle through a 
celestial body intersects the horizon, usually measured clockwise. Sometimes the 
southern point is used as the reference direction, and the measurement is made 
clockwise through 360 degrees." 

Therefore, the amount of yaw is the amount of turn around a vertical axis, while 
the azimuth is the amount of turn around a reference direction. The two terms are 
synonymous in this context, e.g. the amount of yaw deviation, drift, or error, is 
equivalent to amount of azimuth deviation, drift, or error. 

Thus, it is inherent that the use of a gyroscopic tracking system in the invention 
of Chen and Schneider in view of Rallison will accumulate an error in the measured 
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directions, Including the azimuth measurements. Therefore, the use of the yaw 
correction Information in the matrix transformations in the invention of Chen and 
Schneider in view of Rallison is equivalent to the use of the azimuth connection 
information. 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. SeeMPEP 
§ 706.07(a). Applicant Is reminded of the extension of time policy as set forth In 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 . 1 36(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications firom the 
examiner should be directed to Eric V. Woods whose telephone number is 571-272- 
7775. The examiner can normally be reached on M-F 7:30-4:30 alternate Fridays off. 
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If attempts to reach tlie examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi can be reached on 571-272-7664. The fax phone number 
for the organization where this application or proceeding is assigned is 671-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Eric Woods 




12 September 2005 



.-'R!P/«ARYEXAMiNER 



